Physics-Informed Deep Equilibrium Models for Solving ODEs

TCC(graduação) - Universidade Federal de Santa Catarina. Centro Tecnológico. Engenharia de Controle e Automação.Redes neurais para solução de problemas físicos, denominadas physics-informed neural networks (PINNs), e modelos profundos de equilíbrio (do inglês, DEQs), são contribuições recentes que facilitam o uso de modelos de aprendizagem profunda, conhecidos pela capacidade representativa, de aplicações com requisitos realistas de robustez, explicabilidade e escassez de dados. PINNs se mostraram uma forma eficiente de treinar redes neurais para modelar fenômenos físicos. DEQs, por outro lado, empregam uma nova arquitetura que promete mais capacidade de representação com menos parâmetros. Este trabalho consiste em um estudo de ambos, além de uma aplicação que combina-os para resolver problemas de valor inicial de equações diferenciais ordinárias, com um modelo chamado PIDEQ. A abordagem proposta para resolver esse tipo de problema foi implementada e testada utilizando o oscilador de Van der Pol, com uma análise de impacto dos seus diferentes hiperparâmetros. Os resultados mostram que, de fato, é possível treinar um PIDEQ para resolver o problema proposto, gerando soluções aproximadas com baixo erro.Physics-informed neural networks (PINNs) and deep equilibrium models (DEQs) are novel approaches that approximate deep learning’s representational power to applications with realistic requirements such as robustness, data scarcity and explainability. PINNs propose an efficient way to train neural networks to model physical phenomena. DEQs are a new model architecture that can provide more representational power with fewer parameters. This work aims to study both and apply them to solve initial-value problems (IVPs) of ordinary differential equations (ODEs), in an approach coined physics-informed deep equilibrium model (PIDEQ). We implement the proposed approach and test it, analyzing the impacts of the multiple hyperparameters in the approximate solution of the Van der Pol oscillator. Our results show that indeed PIDEQ models are able to solve IVPs, providing approximate solutions with small errors

Deep-learning-based Early Fixing for Gas-lifted Oil Production Optimization: Supervised and Weakly-supervised Approaches

Maximizing oil production from gas-lifted oil wells entails solving Mixed-Integer Linear Programs (MILPs). As the parameters of the wells, such as the basic-sediment-to-water ratio and the gas-oil ratio, are updated, the problems must be repeatedly solved. Instead of relying on costly exact methods or the accuracy of general approximate methods, in this paper, we propose a tailor-made heuristic solution based on deep learning models trained to provide values to all integer variables given varying well parameters, early-fixing the integer variables and, thus, reducing the original problem to a linear program (LP). We propose two approaches for developing the learning-based heuristic: a supervised learning approach, which requires the optimal integer values for several instances of the original problem in the training set, and a weakly-supervised learning approach, which requires only solutions for the early-fixed linear problems with random assignments for the integer variables. Our results show a runtime reduction of 71.11% Furthermore, the weakly-supervised learning model provided significant values for early fixing, despite never seeing the optimal values during training.Comment: Paper accepted at SBAI 202

Towards fully automated deep-learning-based brain tumor segmentation: is brain extraction still necessary?

State-of-the-art brain tumor segmentation is based on deep learning models applied to multi-modal MRIs. Currently, these models are trained on images after a preprocessing stage that involves registration, interpolation, brain extraction (BE, also known as skull-stripping) and manual correction by an expert. However, for clinical practice, this last step is tedious and time-consuming and, therefore, not always feasible, resulting in skull-stripping faults that can negatively impact the tumor segmentation quality. Still, the extent of this impact has never been measured for any of the many different BE methods available. In this work, we propose an automatic brain tumor segmentation pipeline and evaluate its performance with multiple BE methods. Our experiments show that the choice of a BE method can compromise up to 15.7% of the tumor segmentation performance. Moreover, we propose training and testing tumor segmentation models on non-skull-stripped images, effectively discarding the BE step from the pipeline. Our results show that this approach leads to a competitive performance at a fraction of the time. We conclude that, in contrast to the current paradigm, training tumor segmentation models on non-skull-stripped images can be the best option when high performance in clinical practice is desired.Comment: 15 pages, 9 figure

A Graph Neural Network Approach to Nanosatellite Task Scheduling: Insights into Learning Mixed-Integer Models

This study investigates how to schedule nanosatellite tasks more efficiently using Graph Neural Networks (GNN). In the Offline Nanosatellite Task Scheduling (ONTS) problem, the goal is to find the optimal schedule for tasks to be carried out in orbit while taking into account Quality-of-Service (QoS) considerations such as priority, minimum and maximum activation events, execution time-frames, periods, and execution windows, as well as constraints on the satellite's power resources and the complexity of energy harvesting and management. The ONTS problem has been approached using conventional mathematical formulations and precise methods, but their applicability to challenging cases of the problem is limited. This study examines the use of GNNs in this context, which has been effectively applied to many optimization problems, including traveling salesman problems, scheduling problems, and facility placement problems. Here, we fully represent MILP instances of the ONTS problem in bipartite graphs. We apply a feature aggregation and message-passing methodology allied to a ReLU activation function to learn using a classic deep learning model, obtaining an optimal set of parameters. Furthermore, we apply Explainable AI (XAI), another emerging field of research, to determine which features -- nodes, constraints -- had the most significant impact on learning performance, shedding light on the inner workings and decision process of such models. We also explored an early fixing approach by obtaining an accuracy above 80\% both in predicting the feasibility of a solution and the probability of a decision variable value being in the optimal solution. Our results point to GNNs as a potentially effective method for scheduling nanosatellite tasks and shed light on the advantages of explainable machine learning models for challenging combinatorial optimization problems

Does pre-training on brain-related tasks results in better deep-learning-based brain age biomarkers?

Brain age prediction using neuroimaging data has shown great potential as an indicator of overall brain health and successful aging, as well as a disease biomarker. Deep learning models have been established as reliable and efficient brain age estimators, being trained to predict the chronological age of healthy subjects. In this paper, we investigate the impact of a pre-training step on deep learning models for brain age prediction. More precisely, instead of the common approach of pre-training on natural imaging classification, we propose pre-training the models on brain-related tasks, which led to state-of-the-art results in our experiments on ADNI data. Furthermore, we validate the resulting brain age biomarker on images of patients with mild cognitive impairment and Alzheimer's disease. Interestingly, our results indicate that better-performing deep learning models in terms of brain age prediction on healthy patients do not result in more reliable biomarkers.Comment: Accepted at BRACIS 202

Construction and calibration of Time Domain Reflectometry probes for assessing soil humidity in distropheric red latosol

Among the indirect methods of assessing soil moisture, Time Domain Reflectometry (TDR) stands out, which uses the soil dielectric constant to provide volumetric moisture efficiently, quickly and non-destructively. Despite a practical and precise method, TDR has a high cost due to the probes and its Data Logger. In view of this, the present work aims to build and calibrate TDR probes to assess moisture in a Dystrophic Red Latosol. The present work was carried out in the experimental area of the hydraulics laboratory of the Federal University of Grande Dourados (UFGD), located in Dourados-MS, at latitude 22⁰ 12 \u27south, longitude 54⁰ 59\u27 west and altitude of 434 meters. Each probe built consisted of 3 stainless steel rods (Ø = 3 mm; L = 230 mm) RG 98 cable with 90% mesh and 50 ohm impedance, 4.7 pF ceramic capacitor and BNC connector. The construction procedures followed the following steps: 1- Making the cable, 2- Preparing the rods, 3- Welding the rods to the wires, 4 -Operating test and 5 - finishing phase. After construction, they were calibrated with the characteristic soil of the Region, proceeding with the Probe Reading in two depths (10 and 30 cm) and simultaneous collection of deformed soil samples to determine the moisture based on mass in Laboratory. Subsequently, calibrations with cubic polynomial adjustment were performed. The results showed adjustments with high determination coefficients, and the probes developed showed satisfactory performances

Optimization of zirconia surface textured designs using Nd:Yag laser for biomedical applications

The development of surface textured designs has influence in primary stability of surgically placed implants since a textured surface allows to firmer mechanical link to the surrounding tissue. Laser technology has been investigated to develop new surface designs on green zirconia compacts by cold pressing. Nd:Yag laser were used to produce several strategies and different laser parameters (laser power, speed and laser passages) were tested to evaluate their impact on cavities geometry and depth. The surface texture designs were analysed by Scanning Electron Microscopy (SEM) and regular geometries such as cavities or pillars were observed. The distance between lines have a strong impact on texturing quality and should be combined with optimum power and speed conditions. Regarding the optimized conditions, several surface textured patterns were created in both green and sintered zirconia compacts. This study allowed to conclude that only some texturing strategies are suitable to obtain high quality surface textured patterns. Otherwise, the remaining strategies are potential solutions for obtaining high quality machined structures (laser does not machine cavities but crosses the entire bulk). High strength zirconia scaffolds were machined by laser and CNC machining technologies and the two promising technologies were compared.This work is supported by FCT (Fundação para a Ciência e a Tecnologia) through the grant SFRH/BD/148031/2019, the project UIDB/04436/2020 and UIDP/04436/2020

Beekeeping in Brazil: A Bibliographic Review

Brazil presents favorable conditions for beekeeping, having a suitable climate; native flowering plants with great potential for the production of honey, pollen, propolis, and royal jelly; and bees adapted to our conditions, tolerant to the main apicultural diseases and highly productive. Through the meliponiculture, the conservation of mainly native forest areas is allowed; therefore, they are the best environments for the creation of native bees and production of by-products of the beehive with quality. The stingless bees are very sensitive to any disturbance due to anthropogenic action. A systematic bibliographical review was carried out in different electronic databases, through descriptors referring to beekeeping in Brazil. The identification of articles and their inclusion occurred between January 2018 and April 2018. The bibliographic research was conducted in the following electronic databases: (1) Scientific Electronic Library Online (SciELO), (2) Public Library of Science (PLOS) Biology, and (3) ScienceDirect. In order to help in the process of standardization of bee products and traceability of the production chain, it was possible to draw a profile of the main bioactive substances of the beehive products of Brazil. It was also possible to relate the benefits of an adequate management of beekeeping and meliponiculture in Brazil

Measurement of the cosmic ray spectrum above 4×10184{\times}10^{18} eV using inclined events detected with the Pierre Auger Observatory

A measurement of the cosmic-ray spectrum for energies exceeding $4{\times}10^{18}$ eV is presented, which is based on the analysis of showers with zenith angles greater than $60^{\circ}$ detected with the Pierre Auger Observatory between 1 January 2004 and 31 December 2013. The measured spectrum confirms a flux suppression at the highest energies. Above $5.3{\times}10^{18}$ eV, the "ankle", the flux can be described by a power law $E^{-\gamma}$ with index $\gamma=2.70 \pm 0.02 \,\text{(stat)} \pm 0.1\,\text{(sys)}$ followed by a smooth suppression region. For the energy ($E_\text{s}$) at which the spectral flux has fallen to one-half of its extrapolated value in the absence of suppression, we find $E_\text{s}=(5.12\pm0.25\,\text{(stat)}^{+1.0}_{-1.2}\,\text{(sys)}){\times}10^{19}$ eV.Comment: Replaced with published version. Added journal reference and DO